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Abstract 

C*3 ! 

O . We use the idea of dependence balance [1] to obtain a new outer bound for the ca- 

pacity region of the discrete memoryless multiple access channel with noiseless feedback 
(MAC-FB). We consider a binary additive noisy MAC-FB whose feedback capacity is 
not known. The binary additive noisy MAC considered in this paper can be viewed 
as the discrete counterpart of the Gaussian MAC-FB. Ozarow [2] established that the 
capacity region of the two-user Gaussian MAC-FB is given by the cut-set bound. Our 
result shows that for the discrete version of the channel considered by Ozarow, this is 
not the case. Direct evaluation of our outer bound is intractable due to an involved 

O ■ 

auxiliary random variable whose large cardinality prohibits an exhaustive search. We 
overcome this difficulty by using functional analysis to explicitly evaluate our outer 
bound. Our outer bound is strictly less than the cut-set bound at all points on the 
capacity region where feedback increases capacity. In addition, we explicitly evaluate 
the Cover-Leung achievable rate region [3] for the binary additive noisy MAC-FB in 
consideration. Furthermore, using the tools developed for the evaluation of our outer 
bound, we also explicitly characterize the boundary of the feedback capacity region of 
the binary erasure MAC, for which the Cover-Leung achievable rate region is known 
to be tight. This last result confirms that the feedback strategies developed in [4] for 
the binary erasure MAC are capacity achieving. 



*This work was supported by NSF Grants CCF 04-47613, CCF 05-14846, CNS 07-16311 and CCF 07- 
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1 Introduction 



Noiseless feedback can increase the capacity region of the discrete memory less MAC, unlike 
for the single-user discrete memoryless channel. This was shown by Gaarder and Wolf 
in [5] for the binary erasure MAC, which is defined as Y = X 1 + X 2 . Ozarow showed 
in [2] that feedback can also increase the capacity region of a two-user Gaussian MAC-FB. 
A constructive achievability scheme based on the classical Kailath-Schalkwijk [6] feedback 
scheme was shown to be optimal for the two-user Gaussian MAC-FB. Moreover, the cut-set 
outer bound was shown to be tight in this case. 

Subsequently, Cover and Leung obtained an achievable rate region for the general MAC- 
FB based on block Markov superposition coding [3]. Even though this region is in general 
larger than the capacity region of the MAC without feedback, it is not optimal for the 
two-user Gaussian MAC-FB, as was shown in [2]. Kramer [7] used the notion of directed 
information to obtain an expression for the capacity region of the discrete memoryless MAC- 
FB. Unfortunately, this expression is in an incomputable non-single-letter form. Recently, 
Bross and Lapidoth [8] proposed an achievable rate region for the two-user discrete memo- 
ryless MAC-FB and showed that their region includes the Cover-Leung region, the inclusion 
being strict for some channels. 

For a specific class of MAC-FB, Willems [9] developed an outer bound that equals the 
Cover-Leung achievable rate region. For this class of MAC-FB, each channel input (say 
X\) should be expressible as a deterministic function of the other channel input (X2) and 
the channel output (Y). The binary erasure MAC considered by Gaarder and Wolf, where 
Y — Xi + X 2 , falls into this class of channels. Therefore, Cover-Leung region is the feedback 
capacity region for the binary erasure MAC. 

A general outer bound for MAC-FB is the cut-set bound. Although the cut-set bound was 
shown to be tight for the two-user Gaussian MAC-FB, it is in general loose. An intuitive 
reason for the cut-set bound to be loose for the general MAC-FB is its permissibility of 
arbitrary input distributions, some of which yielding rates which may not be achievable. For 
instance, even though Cover-Leung achievability scheme introduces correlation between X\ 
and X2, it is a limited form of correlation, as the channel inputs are conditionally independent 
given an auxiliary random variable, whereas the cut-set bound allows all possible correlations. 

The idea of dependence balance was introduced by Hekstra and Willems in [1] to obtain 
an outer bound on the capacity region of the single-output two-way channel. The basic idea 
behind this outer bound is to restrict the set of allowable input distributions, consequently 
restricting arbitrary correlation between channel inputs. The authors also developed a par- 
allel channel extension for the dependence balance bound. The parallel channel extension 
can be interpreted as follows: the parallel channel output can be considered as a genie aided 
information which is made available at both transmitters and the receiver and it also effects 
the set of allowable input distributions through the dependence balance bound. Depending 
on the choice of the genie information (which is equivalent to choosing a parallel channel), 
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there is an inherent tradeoff between the set of allowable input distributions and the exces- 
sive mutual information rate terms which appear in the rate expressions as a consequence 
of the parallel channel output. We will exploit this tradeoff provided by the parallel channel 
extension of the dependence balance bound to obtain a strict improvement over the cut-set 
bound for a particular MAC whose feedback capacity is not known. 

To motivate the choice of our MAC, consider the binary erasure MAC used by Gaarder 
and Wolf given by Y = X\ + X 2 . If we introduce binary additive noise at the channel output, 
then the channel becomes Y = X\ + X2 + N, where all Xi, X2 and N are binary and N 
has a uniform distribution. This is a non-deterministic noisy MAC which does not fall into 
any class of channels for which the feedback capacity is known. We should mention that this 
particular MAC was extensively studied by Kramer in [7,10], where the first improvement 
over the Cover-Leung achievable rate region was obtained. 

We extend the idea of dependence balance to obtain an outer bound for the entire capacity 
region of this binary additive noisy MAC-FB. Direct evaluation of the parallel channel based 
dependence balance bound is intractable due to an involved auxiliary random variable whose 
large cardinality prohibits an exhaustive search. We use composite functions and their 
properties to obtain a simple characterization for our bound. Our outer bound strictly 
improves upon the cut-set bound at all points on the boundary where feedback increases 
capacity. In addition, we explicitly evaluate the Cover-Leung achievable rate region for our 
binary additive noisy MAC-FB. 

We particularly focus on the symmetric-rat ^\ point on the feedback capacity region of 
this channel. Cover-Leung's achievable symmetric-rate for this channel was obtained in [10] 
as 0.43621 bits/transmission. In [10], Kramer obtained an improved symmetric-rate inner 
bound as 0.43879 bits/transmission by using superposition coding and binning with code 
trees. The cut-set upper bound on the symmetric-rate was obtained in [10] as 0.45915 
bits/transmission. We obtain a symmetric-rate upper bound of 0.45330 bits/transmission 
which strictly improves upon the cut-set bound. Furthermore, we also show that a binary 
and uniform selection of the involved auxiliary random variable is sufficient to obtain our 
symmetric-rate upper bound. 

It should be remarked that the channel we consider in this paper can be thought of as the 
discrete counterpart of the channel considered by Ozarow [2]. Although the cut-set bound 
was shown to be tight for the two-user Gaussian MAC-FB, our result shows that the cut-set 
bound is not tight for the discrete version of the additive noisy MAC-FB. 

As an application of the properties of the composite functions developed in this paper, 
we are able to obtain the entire boundary of the capacity region of the binary erasure MAC- 
FB. The evaluation of the asymmetric rate pairs on the boundary of the feedback capacity 
region of the binary erasure MAC was mentioned as an open problem in [11]. It was shown 

1 By symmetric-rate point, we refer to the maximum rate R such that the rate pair (R, R) lies in the 
capacity region of MAC-FB. 
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in [12] that a binary and uniform auxiliary random variable T is sufficient to attain the 
sum-rate point on the capacity region of the binary erasure MAC-FB. We show here that 
this is also the case for any asymmetric rate point on the boundary of the feedback capacity 
region. This result also complements the work of Kramer [4] , where feedback strategies were 
developed for the binary erasure MAC-FB and it was shown that these strategies achieve 
all rates yielded by a binary selection of the auxiliary random variable T in the capacity 
region. Our result hence shows in effect that the feedback strategies developed in [4] for 
binary erasure MAC are optimal and capacity achieving. 

2 System Model 

A discrete memoryless two-user MAC-FB (see Figure 1) is defined by the following: two 
input alphabets X\ and X 2 , an output alphabet y, and the channel defined by a probability 
transition function p(y\xi, x 2 ) for all (xi, x 2 , y) G X\ x X 2 x y. A (n, Mi, M 2 , P e ) code for the 
MAC-FB consists of two sets of encoding functions fu, f 2 { for i — 1, . . . , n and a decoding 
function g 

fii-.Mixy-'^Xi, i = l,...,n 
fx ■ M 2 x y- 1 - X 2 , i = l,...,n 
g-.y n ^M 1 x M 2 

The two transmitters produce independent and uniformly distributed messages W\ G {1, . . . , 
Mi} and W 2 G {1, . . . ,M 2 }, respectively, and transmit them through n channel uses. The 
average error probability is defined as P e = Pr(g(Y n ) ^ (Wi, W 2 )). A rate pair \Rii R2) is 
said to be achievable for MAC-FB if for any e > 0, there exists a pair of n encoding functions 
{/ii}r=i, {/2i}f=i, and a decoding function g such that Ri < \og(M 1 )/n : R 2 < \og(M 2 )/n 
and P e < e for sufficiently large n. The capacity region of MAC-FB is the closure of the set 
of all achievable rate pairs (Ri,R 2 ). 

3 Cut-Set Outer Bound for MAC-FB 

By applying Theorem 14.10.1 in [13], the cut-set outer bound on the capacity region of 
MAC-FB can be obtained as: 

CS = {(Ri,R 2 ) : Ri < I{Xi-Y\X 2 ) (1) 
R 2 <I(X 2 ;Y\Xi) (2) 
Ri + R2<I(Xi,X 2 ;Y)} (3) 



4 



\v 2 





Encoder 1 


x 1 






1 










Encoder 2 




1 




x 2 



T 



Y 



Decoder 



Figure 1: The multiple access channel with noiseless feedback (MAC-FB). 

where the random variables (Xi, X 2 ,Y) have the joint distribution 

p(xi, x 2 , y) = p(xi, x 2 )p(y\xi, x 2 ) 



(4) 



The cut-set outer bound allows all input distributions p(x\,x 2 ), which makes it seemingly 
loose since an achievable scheme might not achieve arbitrary correlation and rates given by 
the cut-set bound. Our aim is to restrict the set of allowable input distributions by using a 
dependence balance approach. 

4 Dependence Balance Outer Bound for MAC-FB 

Hekstra and Willems [1] showed that the capacity region of MAC-FB is contained within 
VB, where 



VB = [{R X ,R 2 ) : R, < J(X i; r|A 2 ,T) 
R 2 <I(X 2 ;Y\X 1 ,T) 
Ri + R 2 <I(X 1 ,X 2 ;Y)} 

where the random variables (Xi, X 2 ,Y,T) have the joint distribution 

pit, xi,x 2 , y) = p(t)p(x 1} x 2 \t)p(y\x 1} x 2 ) 

and also satisfy the following dependence balance bound 

I(X 1 ;X 2 \T)<I(X 1 ;X 2 \Y,T) 



(5) 
(6) 
(7) 



(9) 



where T is subject to a cardinality constraint of |T| < |A!i||A?2| +2. The dependence balance 
bound restricts the set of input distributions in the sense that it allows only those input 
distributions p(t, Xi,x 2 ) which satisfy ([9]). It should be noted that by ignoring the constraint 
in (El), one obtains the cut-set bound. 
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5 Adaptive Parallel Channel Extension of the Depen- 
dence Balance Bound 



In [1], Hekstra and Willems also developed an adaptive parallel channel extension for the 
dependence balance bound which is given as follows: Let A(ZY) denote the set of all distri- 
butions of U and A(U\V) denote the set of all conditional distributions of U given V. Then 
for any mapping F : A(X% x X 2 ) — > A(Z\Xi x X 2 x y), the capacity region of the MAC-FB 
is contained in 

VB P c = {{Ri,R*) : Ri < I(X i; Y,Z\X 2 ,T) (10) 

R 2 <I(X 2 ;Y,Z\X 1 ,T) (11) 

R 1 <I(X 1 ;Y\X 2 ) (12) 

R 2 <I(X 2 ;Y\X 1 ) (13) 

Rx + R*<UX X ,X 2 ;Y) (14) 

R 1 + R 2 <I(X 1 ,X 2 ;Y,Z\T)} (15) 

where the random variables (Xi,X 2 , Y, Z, T) have the joint distribution 

p(t, x lt x 2 , y, z) = p(t)p(x 1 , x 2 \t)p(y\x 1 x 2 )p + (z\x 1 , x 2 , y, t) (16) 

such that for all t 

p + (z\x u x 2 ,y,t) = F(p XlX2 (x 1 ,x 2 \t)) (17) 

and such that 

I(X 1 -,X 2 \T)<I(X 1 ;X 2 \Y,Z,T) (18) 

where T is subject to a cardinality bound of |T| < |A4||Af 2 | + 3. 

We should remark that the parallel channel (defined by p + (z\xi, x 2 , y, t)) is selected apri- 
ori, and for every choice of the parallel channel, one obtains an outer bound on the capacity 
region of MAC-FB, which is in general tighter than the cut-set bound. The set of allowable 
input distributions p(t,xi,x 2 ) are those which satisfy the constraint in (fT8j) . Also note that 
only the right hand side of (TT8l) . i.e., only I(Xi, X 2 \Y, Z, T), depends on the choice of the 
parallel channel. By carefully selecting p + (z\x±, x 2 , y, t), one can reduce I(Xi; X 2 \Y, Z,T), 
thereby making the constraint in (j!8p more stringent, consequently reducing the set of al- 
lowable input distributions. To obtain an improvement over the cut-set bound, we need 
to select a "good" parallel channel such that it restricts the input distributions to a small 
allowable set and yields small values of I{X\] Z\Y, X 2 ,T) and I(X 2 ; Z\Y, X±, T) at the same 
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time. These two mutual information "leak" terms are the extra terms that appear in (jTUl) 
and (fTTj) relative to the rates appearing in ([5]) and ([6]), respectively. 

To motivate the choice of our particular parallel channel, first consider a trivial choice of 
Z: Z = (f> (a constant). For this choice of Z, (|T8l) reduces to ([9]) and we are not restricting the 
set of allowable input distributions any more than the T>B bound. Moreover, for a constant 
selection of Z, ffTUj) and fTTTj) reduce to ([5]) and §6§, respectively. Thus, a constant selection 
of Z for VBpc is equivalent to VB itself. 

Also note that the smallest value of I{X\\ X 2 \Y, Z, T) is zero. Thus, it follows that if 
we select a parallel channel such that I(X±; X2\Y, Z, T) = for every input distribution 
p(t,Xi,x 2 ), then J(X 1 ;X 2 |T) = by (fIBl) . Hence, the smallest set of input distributions 
permissable by VBpc consists of those p(t,Xi,x 2 ) for which X x and X 2 are conditionally 
independent given T. Furthermore, for a parallel channel such that I(Xi, X 2 \Y, Z, T) = 0, 
the bound in ( IT51) is redundant. This can be seen from: 

= /(Xi; X 2 |T) - I(X i; X 2 % Z, T) 
= J(Xi; F, Z|T) - I{X l ; Y, Z\X 2 , T) 

= I(X 1 ,X 2 ; Y, Z\T) - I(X 1 ; Y, Z\X 2 , T) - I(X 2 ; Y, Z\X U T) (19) 

Using (fT9~l) . it is clear that the sum of constraints (TTUT) and (TTTT) is at least as strong as the 
constraint ( fT5l) . This shows that (JT5l) is redundant for the class of parallel channels where 
I(X 1 ;X 2 \Y,Z,T) = 0. 

6 Binary Additive Noisy MAC-FB 

In this paper, we will consider a binary-input additive noisy MAC given by 

Y = X 1 +X 2 + N (20) 

where N is binary, uniform over {0, 1} and is independent of X\ and X 2 . The channel 
output Y takes values from the set 3^ = {0, 1, 2, 3}. This channel does not fall into any class 
of MAC for which the feedback capacity region is known. This channel was also considered 
by Kramer in [7, 10] where it was shown that the Cover-Leung achievable rate is strictly 
sub-optimal for the sum-rate. 

We select a parallel channel p + (z\xi,x 2 ,y) such that I(Xi\ X 2 \Y, Z, T) = 0. By (|T8|) . 
this will imply I(Xi, X 2 \T) = 0, and hence only distributions of the type p(t,xi,x 2 ) = 
p(t)p(xi\t)p(x 2 \t) will be allowed. By doing so, we restrict the set of allowable input dis- 
tributions to be the smallest permitted by VBpc, although we pay a penalty due to the 
positive "leak" terms I(X X \ Z\Y,X 2 ,T) and I(X 2 ; Z\Y, X x , T). 

Two simple choices of Z which yield I(X 1 ; X 2 \Y, Z, T) = are Z = X x and Z = X 2 . For 
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each of these choices, the corresponding outer bounds are, 

VB% = [{R U R 2 ) : Rt < I(X 1 ;Y\X 2 ,T) + H{X 1 \Y,X 2 ,T) (21) 

R 2 <I(X 2 ;Y\X 1 ,T) (22) 

R 1 <I(X 1 ;Y\X 2 ) (23) 

R 1 + R 2 <I{X h X 2 ;Y)} (24) 

and 

VB% = {(Ri,R 2 ) : Ri < I(X i; Y\X 2 ,T) (25) 

R 2 < I(X 2 ;Y\X U T) + H(X 2 \Y,X 1 ,T) (26) 

R 2 <I{X 2 ;Y\X X ) (27) 

R 1 +R 2 <I(X h X 2 ;Y)} (28) 

where both VB% and VB% are evaluated over the set of input distributions of the form 
p(t,xi,x 2 ) = p(t)p(xi\t)p(x 2 \t). 

For the binary additive noisy MAC-FB in consideration which is given in (1201) . the fol- 
lowing equalities hold for any distribution of the form p(t, Xi, x 2 ) = p(t)p(xi\t)p(x 2 \t) , 

H(X 1 \Y,X 2 ,T) = ±H(X 1 \T) (29) 

H{X 2 \Y,X 1 ,T)= l -H{X 2 \T) (30) 

Using (1291) and (|30|) . we can simplify T>B^p C and UBpg as, 

©i3^= {(i2i,i2 2 ) : i2i<min(/(Xi;y|jr 2 ),H(Xi|T)) (31) 

i? 2 < \h(X 2 \T) (32) 

i? 1 + J R 2 </(X 1 ,X 2 ;y)} (33) 

and 

VB% = [(Rx,R 2 ) : Rx < \h{X x \T) (34) 

i? 2 < min (I(X 2 ; Y\X X ),H(X 2 \T)) (35) 

Ri + R 2 <I(X h X 2 ;Y)} (36) 

where both bounds are evaluated over the set of distributions of the form p(t, xi,x 2 ) = 
p(t)p(xi\t)p(x 2 \t) and the auxiliary random variable T is subject to a cardinality constraint 
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of \T\ < \Xx\\X%\ + 3. The evaluation of the above outer bounds is rather cumbersome 
because for binary inputs, the bound on \T\ is \T\ < 7. To the best of our knowledge, no 
one has been able to conduct an exhaustive search over an auxiliary random variable whose 
cardinality is larger than 4. In Section 8, we will obtain an alternate characterization for our 
outer bounds using composite functions and their properties. For that, we will first develop 
some useful properties of composite functions in the next section. 

A valid outer bound is given by the intersection of T>B^ C and VB^q, 

VB PC = VB%f)VB% (37) 

We will show that this outer bound is strictly smaller than the cut-set bound at all points 
on the capacity region where feedback increases capacity. 



7 Composite Functions and Their Properties 

Before obtaining a characterization of our outer bounds, we will define a composite function 
and prove two lemmas regarding its properties. These lemmas will be essential in obtaining 
simple characterizations for our outer bounds and the Cover-Leung achievable rate region. 
Throughout the paper, we will refer to the entropy function as h^ k \s\, . . . S&) which is defined 



h^(8 U ... ) 8 k ) = -^2 8i l0g{8 i ) (38) 



i=l 



for Sj > 0, i = 1 . . . , k, and Yli=i s i = 1> where all logarithms are to the base 2. We will 
denote h^ 2 '(s) simply as h(s). To characterize our bounds, we will make use of the following 
function 



(.) = \ fg- fM £ ^ l ' 2 (39) 
^ for 1/2 < s < i 



It was shown in [12] that the composite function h(4>(s)) is symmetric around s = 1/2 and 
concave in s for < s < 1. The functions 0(s) and h(4>(s)) are illustrated in Figure 2. From 
the definition of <f>(s) in (|39|) it is clear that for any s G [0, 1], the function <j>(s) satisfies the 
following property 

0(2s(l - s)) = min(s, 1 - s) (40) 
As a consequence, the following holds as well 

h{(j)(2s{l - a))) = h(s) (41) 
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Figure 2: Functions 0(s) and h(</>(s)). 

For any s G [0, 1], the following holds from the definition of 0(s), 

0(2s(l-s)), 0<s<| 



(42) 

l-0(2s(l-s)), | < s < 1 

For any x G [0, |] and w G [0, |], let us define a function 

f(x,y)±<f>(x)+<i>(y)-2<!>(x) ( i>(y) (43) 
= l-V(l-2x)(l- 2y ) (44) 

From the above definition, it is clear that the function f(x,y) lies in the range [0, |]. 
Lemma 1 TTie variable 

v = si + s 2 - 2sis 2 (45) 
zs always lower bounded by f(2s 1 (l — si), 2s 2 (l — s 2 )) /or any Si G [0, 1], s 2 G [0, 1]. 

Proof: We will prove this lemma by considering all four possible cases. 

1. If si G [0, §], s 2 G [0, §], then from ggj, Sl = 0(2 Sl (l - s x )), s 2 = 0(2s 2 (l - s 2 )) and 
hence 

u = /(2si(l-ai),2s 2 (l-s 2 )) (46) 

2. If Sl G [|,l],s 2 G [|,1], then from (02J, Sl = 1 - 0(2^(1 - s 2 = 1 - 0(2s 2 (l - s 2 )) 



10 



and hence 

v = f(2s 1 (l-s 1 ),2s 2 (l-s 2 )) (47) 

3. If si G [0, §], s 2 G [§, 1], then from QgJ, Sl = 0(2^(1 - Sl )), s 2 = 1 - 0(2s 2 (l - s 2 )) 
and hence 

U = 1-/(231(1-50,252(1-52)) 

> /(2 Sl (l-sO,2s 2 (l-s 2 )) (48) 
where (a) follows by the fact that /(2si(l — Si),2s 2 (l — s 2 )) < \- 

4. If si G [§, 1], s 2 G [0, 1], then from (g2J|, si = 1 - 0(2^(1 - s x )), s 2 = 0(2s 2 (l - s 2 )) 
and hence 

u = l-/(2 Sl (l-sO>2s 2 (l-S2)) 

(6) 

>/(2si(l-si),2s a (l-s a )) (49) 
where (6) follows by the fact that /(2si(l — Si),2s 2 (l — s 2 )) < ~. 

Thus, for any pair (si,s 2 ), where Si G [0, 1], s 2 G [0, 1], we have shown that v > /(2si(l — 
s0,2s 2 (l-s 2 )). ■ 

Lemma 2 TTie function f(x,y) is jointly convex in (x, y) for < x < §,0 < y < \. 

Proof: Showing that the function f(x,y) is jointly convex in (x,y) is equivalent to showing 
that the Hessian matrix, H of f(x, y) is positive semi-definite, which is equivalent to showing 
that the eigenvalues of H are non-negative. The Hessian matrix, H, of f(x,y) is 

vg=2y -1 



H= | 2(1 -_T 2 2 v / ^gp J (50) 

2 v /(l-2x)(l-2y) 2(l-2tf)»/a 

The two eigenvalues of are 

Ai = 

2 2^(1 -2a;) 3 / 2 (1 - 2y) 3 / 2 y | 1 J 

which are non- negative for all < x < \ and < y < |, thus completing the proof. ■ 
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8 Evaluation of the Dependence Balance Outer Bound 

We will now return to the characterization of our upper bounds T>B^ C and T>Bp%. Let the 
cardinality of the auxiliary random variable T be fixed and arbitrary, say \T\. Then, the 
joint distribution p{t)p{x 1 \t)p{x 2 \t) can be described by the following variables: 

q lt = Pr(X 1 = 0\T = t), t = l,...,\T\ 
q 2t = Pr(X 2 = 0\T = t), t = l,...,|T| 

Pt = Pi(T = t), t = l,...,\T\ (52) 

We will characterize our outer bounds in terms of three variables ui, u 2 and u which are 
functions of p(t, x±, x 2 ), and are defined as, 

ui = y]ptgu(i - Qit) = y~]ptuu (53) 

t t 

U2 = y^Ptg2t(l - Q2t) = ^2ptU 2 t (54) 

t t 
^2,vMit + Q2t - 2q lt g 2t ) = ^p t w t (55) 



u 



t 



where we have defined 

uu = <?it(l - git) (56) 

U2t = 92t(l - Q2t) (57) 

Ut = qit + Q2t - 2qitQ2t (58) 

It should be noted that since < qj t < 1, for j — 1, 2, t — 1, . . . , |T|, the variables ui, u 2 , u u 
and u 2t all lie in the range [0, |]. Our outer bounds T>B^p C and VBp^ are comprised of the 
following information theoretic entities: 

1. #(Xi|T), #(X 2 |T) 

2. /(x i; r|x 2 ), /(x^rixo 

3. /(x^x^r). 

We will first obtain upper bounds for each one of these entities individually in terms of 
(ui,u 2 ,u). 
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We upper bound H(Xi\T) as follows, 

H(X 1 \T) = J2pMqu) (59) 

t 

= J2pM<P(2q lt (l-qit))) (60) 

t 

= Y,PtK<f>(2u u )) (61) 
t 

< /i(0(2mi)) (62) 



where ( 1601) follows due to ( 14T1) . ( 16TI) follows from ( 1561) . and ( 162]) follows from the fact that 

h((p(s)) is concave in s and the application of Jensen's inequality [13]. Using a similar set of 
inequalities for H(X 2 \T), we obtain 

H(X 2 \T)<h(<f)(2u 2 )) (63) 

We will now upper bound I{X\\ Y\X 2 ) in terms of the variable u. For this purpose, let 
us first define 

a = P Xl x 2 (0, 0) = S ^p t quq2t (64) 
t 

b = P Xl xM 1) = 5>?i t (l - 92t ) (65) 

i 

c = P M (1, 0) = J^M 1 ~ 9w)9a (66) 

d = Py lX3 (l,l) = l-a-6-c. (67) 



We now proceed as, 



J(X i; F|X 2 ) = H{Y\X 2 )-H{Y\X X ,X 2 ) (68) 
= #(F|X 2 ) - 1 (69) 
ale 



a 



+ c)h {3) 



2(a + c)' 2' 2(a + c) 

^^^(^■i-^)- 1 (70) 

!,»+£)_, (71 ) 

= i/»(6 + c) (72) 

= \h{u) (73) 
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where (1TT1) follows by the concavity of the entropy function and the application of Jensen's 
inequality [13]. Using a similar set of inequalities, we also have 

I(X 2 ;Y\X 1 )<h(u) (74) 

We will now obtain an upper bound on I(X 1 ,X 2 ; Y). First note that 

I(X 1 ,X 2 ; Y) = H(Y) - H(Y\X U X 2 ) (75) 
= /^(P y (0),P y (l),P y (2),P y (3)) - 1 (76) 



where 



P Y (0) = J2ptQitQ2t/2 (77) 

t 

P y (l) = J2pt(qit + Q2t - q U q»)/2 (78) 
t 

P Y (2) = J2Pt{l-quq2t)/2 (79) 

t 

P y (3) = J> t (l- to )(l-g 2t )/2 (80) 



Using the following fact, 



(a, (3, 7 , 0) = (a, f3, 7 , 9) + V } (6, 7 , /?, a) (81) 
<^)^,^,^,^ (82) 



= h(a + 6) + h(^J (83) 

= h(l-(/3 + 7)) + l (84) 

where (1521) follows by the concavity of the entropy function and the application of Jensen's 
inequality [13], we now obtain an upper bound on I(Xi,X 2 ; Y) by continuing from ( 1761) . 

I{X U X 2 ; Y) = /i (4) (Py(0), P y (l), P y (2), P y (3)) - 1 (85) 
<h(l- (Py(l) + P y (2))) +h(\)~l (86) 

= h(^) (87) 



where (JHSD follows by (JSJ) and fl57J) follows from the fact that P y (l) + P y (2) = (1 + u)/2 
using (175)) and (1791) . where u is as defined in (155]) . 
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8.1 A Set of Feasible (u\,U2,u): V 

We have obtained upper bounds on the information theoretic entities which comprise our 
outer bounds in terms of three variables Ui, u 2 and u. We will now give a feasible region for 
these triples based on the structures of these variables. First, note that for any q u 6 [0, 1], 
the following holds: u u = <?it(l — qu) < |- Similarly, u 2t = <?2t(l — Q2t) < i- Hence, we have 

< u x < i (88) 
< u 2 < - (89) 

We now obtain a lower bound on u as 

u = Y J PtU t (90) 

t 

>J2Ptf( 2u u^u 2t ) (91) 
t 

Ptu 2 t (92) 

= f(2u u 2u 2 ) (93) 



where (191]) follows by Lemma 1 and f[92|) follows by Lemma 2 and the application of Jensen's 
inequality [13]. We now obtain another lower bound on u, 



u 

t 



= (94) 

t 

= + Q2t ~ 2qitq2t) (95) 

t 

= ^pMu - q\ t + q2t - q\ t + (qu -q2t) 2 ) (96) 
t 

>^Pt{qit ~ q\ t + q2t - qlt) (97) 
t 

= ^Ptqu(l - qu) + ^2ptq2t(l - q2t) (98) 
t t 

= u 1 + u 2 (99) 
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Finally, we obtain an upper bound on u in terms of u x and u 2 , 

u = Y,PtU t (100) 

t 

= ^PMit + Q2t - 2q u q 2t ) (101) 

t 

= £)pt(?« + Q2t - 2q lt q 2t + q\ t + (1 - q 2t f - q\ t - (1 - q 2t f) (102) 
t 

< ^Pt(qit + - 2q lt q 2t + q\ t + (1 - q 2t f - 2g u (l - g 2i )) (103) 
t 

= 1 - (u x + ua) (104) 

where (11031) follows by the inequality qf t + (1 — g 2 <) 2 — 2<?it(l — ?2t)- 
By noting 



/(2. 1 , 2« 2 ) - K + 1*) = W(1 - 4 ; i)(1 " 4 ^ - (wi + 1*) (105) 



;i - 4«i) + (1 - 4u 2 ) - 2^/(1 -4ui)(l-4u 2 ) 



(VI - 4«i - VI - 4m 2 ) 2 



(106) 
(107) 



> (108) 

and using (1931) . we note that the lower bound in (199]) is redundant. Therefore, from (1931 and 
(11041) . we have the following feasible range for the variable u in terms of u x and u 2 , 

f(2u x ,2u 2 ) < u < 1 - (u 1 +u 2 ) (109) 

Combining fjggj) . (j8"9l and (I109p . a set of feasible (ui,u 2 ,u) is given as follows, 

V = {(ui,u 2 ,u) : < ui < i;0 < w 2 < ^;/(2m 1 ,2m 2 ) < u < 1 - (iii + it 2 )} (HO) 

It should be noted that the set V in (11101) may not necessarily be the smallest feasible set 
of all triples (u x ,u 2 ,u). Since we are interested in a maximization over these set of triples, 
a possibly larger set V suffices. 

8.2 A Simple Characterization of T>Bp C and T>Bp C 

Using the upper bounds on H(X X \T), H(X 2 \T), I(X X ; Y\X 2 ), I(X 2 ; Y\X X ) and I(X X ,X 2 ; Y) 
in (|62|) . (IBlfl) . (!73|) . (1741) and (1571) in terms of (mi,m 2 ,m) along with a feasible set of triples 
V in (IllOp . we obtain the following two outer bounds on the capacity region of the binary 
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additive noisy MAC-FB, starting from fTHT]) - and (|34j)-(j36|), 



and 



{ Ul ,u 2 ,u)e.v I ^ 



R 2 < hi(<p(2u 2 )) 

Ri + R2<h[ l —^ \ ) ciir 



T)»(2) 
ISO p C 



|J i (i?i,i?2) : i2i < |^(0(2tt 1 )) 

(ui,U2,m)G'P I 



R 2 <mm(^h(u),h((f>(2u 2 )) 
Ri + R2<h(^] I (112) 



We will plot these outer bounds and their intersection in Figure 4. In next section, we will 
explicitly characterize our upper bounds for the symmetric-rate point on the capacity region 
of the binary additive noisy MAC-FB in consideration. 



9 Explicit Characterization of the Symmetric-rate Up- 
per Bound 

For the binary additive noisy MAC-FB in consideration, it was shown by Kramer [7] that the 
symmetric-rate cut-set bound is 0.45915 bits/transmission. It was also shown in [7] that the 
Cover-Leung achievable symmetric-rate is 0.43621 bits/transmission and it was improved to 
0.43879 bits/transmission by using superposition coding and binning with code trees. For 
completeness and comparison with existing bounds, we will first completely characterize our 
outer bound for the symmetric-rate by providing the input distribution p(t)p(xi\t)p(x2\t) 
which achieves it. By symmetric-rate we mean a rate R such that the rate pair (R, R) lies in 
the capacity region of MAC-FB. For the symmetric-rate, both T>Bp^ c and VB^p C will yield 
the same upper bound. Hence, we will focus on VB%. Using (EH), we are interested in 
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obtaining the largest R over all (u\,U2,u) G V such that 



i2<minQ/i(w) ) /i(0(2wi))^ (113) 
R < ~h(<f>(2u 2 )) (114) 
2R < h (H5) 

We will show that a seemingly weaker version of the above bound will improve upon the 
symmetric-rate cut-set bound. We will also show that the weaker bound is in fact the same 
as the above bound, and its sole purpose is the simplicity of evaluation and insight into the 
input distribution that attains it. We first obtain a weakened version of f II 1 3 j) as 

R < min (-h(u), h{4>{2u x )) j < h(4>(2ui)) (116) 



Next, consider (11151) 



2R<h[- -) (117) 



(118) 



,1 f(2u l) 2u 2 ) , 
< h - - ^ — ^ — ii (119) 



2 2 
2 2 

where (11191) follows from ( 1931) and the fact that the binary entropy function /i(s) is mono- 
tonically increasing in s for s G [0, |]. Combining (I114p . (11161) and (11191) . we are interested 
in the largest R such that 

R< max min (h(<f>(2 Ul )), h(cf)(2u 2 )),h (\ - /(2til ' (120) 
ui,« 2 e[0,|] V 2 2\2 2 J J 

We note that this upper bound on the symmetric-rate depends only on ^! and U2, and 
therefore, we replace the feasible set V with ui,u 2 G [0, |]. 

We know that h((p(s)) is concave in s for s G [0, 1]. Hence, it follows that both h((j)(2ui)) 
and ^h((p(2u 2 )) are concave in wi and u 2 , respectively, and hence concave in the pair (u±, u 2 ). 
We also have the following lemma. 



Lemma 3 The function 

III I! i . //., i ■ ^ 

is monotonically decreasing and jointly concave in the pair (ui,u 2 ) for Ui,u 2 G [0, 4]. 



l u f l-f(2 Ul ,2u 2 ) 
g{u 1 ,u 2 ) = -h( (121) 
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Proof: It suffices to show that for a fixed u 2 , the function g(u\, u 2 ) is monotonically decreas- 
ing in u\. Substituting the value of f(2ui, 2u 2 ), we have 



l u f l- (0(2 Ml ) + <P(2u 2 ) - 20(2^)0(2^)) \ 
fl'(wi) u 2 ) = —h I ] (1.22) 

1. (\ <f){2u 2 ) 0(2n 1 )(l-20(2n 2 )) \ 



2H2- 2 - ~ 2 J < 123 > 

Now using the fact that <p{2s) is increasing in s for s G [0, |], we have that for u x > u\, 
(j)(2u' 1 ) > (j)(2ux). Moreover, the following holds 

<t>(2u 1 )(l-2<P(2u 2 )) > 0(2 Ml )(l-20(2^ 2 )) 

2 ~ 2 1 ' 

since 4>{2u 2 ) < \. Now using the above inequality along with the fact that the binary entropy 
function h(s) is increasing for < s < |, we have that for u 1 > m, 

1 , A _ 0(2^ 2 ) _ 0(2 Ml )(l-20(2 M2 )) \ > 1 /l _ 0(2^ 2 ) _ 0(2^)(l-20(2^ 2 )) \ 

2 V 22 2 y~2v 2 2 2 / 

(125) 

This shows that for a fixed w 2 , the function g(ui,u 2 ) is monotonically decreasing in U\. As 
the function is symmetric in u\ and u 2 , the monotonicity of g(ui,u 2 ) in (1X1,112) follows. 

To show the concavity of g(u\,u 2 ) in the pair (ui,u 2 ), we first note from Lemma 2 that 
f(2ui, 2u 2 ) is jointly convex in the pair (m 1; m 2 ). We define another function 

1-/(2^,2^) 

£(«i,u 2 ) = g ^ Q > 

Note that ^(mi,m 2 ) is jointly concave in the pair (ui,u 2 ). Furthermore, the binary entropy 
function h(s) is concave and nondecreasing for s G [0, |]. Hence, rewriting the function 
g(ui,u 2 ) as a composition of two functions, we obtain 

g(u 1 ,u 2 ) = hi{^{u 1 ,u 2 )) (127) 

From the theory of composite functions [14], we know that a composite function f\(f 2 (s)) 
is concave in s if /i(.) is concave and nondecreasing and f 2 {s) is concave in s. Identifying 
/i(.) with h(.) and f 2 (ui,u 2 ) with £(wi,-u 2 ), the concavity of g(ui,u 2 ) in the pair {ui,u 2 ) is 
established. ■ 

Therefore, all three functions in the min(.) in f)120p are concave in (ui,u 2 ). Invoking the 
fact that the minimum of concave functions is concave, we conclude that the maximum in 
(11201) is unique. We will now show that the unique pair {u\,u^) that attains this maximum 
satisfies the property that h((f)(2ul)) = \h((p(2u* 2 )) = g(u\,u* 2 ). 
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For this purpose, we first characterize those pairs (ui,u 2 ) such that the following holds, 

h{^{2u x )) = \h(<p(2u 2 )) = g{u 1: u 2 ) (128) 
By using (I128p . we obtain two equations for ui and u 2 , as 

Wfix)) = h ( (129) 



2 V3-20(2mi)7 

From (11291) . one can see that 2ui is the unique solution s G [0, |] of the equation 

w»=H™) (131) 

Obtaining the optimal u\ from the above equation is illustrated in Figure 3. The unique 
solutions (ui,u 2 ) of (11291) and fll3Qf) are 

u x = 0.086063, u 2 = 0.218333 (132) 

We will now show that this pair {u\,u 2 ) yields the maximum in (I120|) . 

Returning to the maximization problem (I120p . first denote S as the region of allowable 

(ui,u 2 ), 

S = {{u 1 ,u 2 ):0<u 1 <-;0<u 2 <-} (133) 
Also define a subset of this region 

S = {(«i,m 2 ) : u x G (ui,-];u 2 G (u 2 , -]} (134) 



where (ui,u 2 ) is given by (11321) . We will now show that the pair (ui,u 2 ) yields the solution 
of the maximization problem in (11201) . Consider the following two cases, 

1. If (ui,u 2 ) G S, then by Lemma 3, we have that g(ui,u 2 ) < g(ui,u 2 ), using which we 
obtain, 

min Ui(0(2u!)), ^h(<f)(2u 2 )), g{u x , u 2 )*j < g(u x , u 2 ) < g(u u u 2 ) (135) 

2. If (ui, u 2 ) G S \ S, we either have U\ < «i or u 2 < u 2 or both. Using this along with 
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Figure 3: Characterization of the optimal u\. 



the fact that h(<p(2s)) is monotonically increasing in s for s G [0, -r), we obtain 



min ( h(<f ) (2u 1 )) 1 -h((j)(2u2)),g(u 1 ,u 2 ) ) < h(</>(2ux)) 



(136) 



The above two cases show the following, 



max min ( /i(0(2«i)), \h{4>{2u 2 )) , g{u u u 2 ] 

ui6[0,i],«26[0,i] V 2 



/i(0(2«i)) 



2^(^(2^)) 

W2) 



(137) 

(138) 
(139) 



Thus, the maximum in (j!20p is obtained at (w*, w|) = (tti, u 2 ). We now obtain a distribution 
p(t)p(xi\t)p(x 2 \t) which attains this symmetric-rate upper bound. Fix T to be binary, and 
select the involved probabilities as 



1 

Po = Pi = 2 
q 10 = l- q u = (j)(2ul) 
920 = 1 - 921 = 0(2^2) 



(140) 

(141) 
(142) 



The reason for constructing such an input distribution is that, at this specific distribution, 
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we have the following exact equalities, 



H(X 1 \T) = h(<j>(2uD) (143) 

l -H{X 2 \T) = \h{4>{2ul)) (144) 

h(X 1 ,X 2 ;Y)=g(ul,u* 2 ) (145) 

and we achieve the outer bound we developed with equality. Substituting the values of 
(m*, u 2 ), we obtain a distribution given by, 

Po = Pi = g ( 146 ) 

q w = l- q n = 0.095109 (147) 

q 20 = l- q 21 = 0.322050 (148) 

The above input distribution yields a symmetric-rate of 0.45330 bits/transmission. Moreover, 
the u* corresponding to this distribution is given by 



u 



y~]l>t(Qit + Qzt ~ 2fl,tg2t) ( 149 ) 
f 

= f(2u* l7 2u*) (150) 

= 0.355899 (151) 

where (j!50p is by construction of the input distribution p(t, X\ , x 2 ) and f)15ip is obtained by 
substituting the distribution specified in fl!46)) - (1148j) . Moreover, <j)(2u 2 ) < u* < |, hence we 
also have that 

\h(u*) > lh((f)(2u* 2 )) = h{<j>{2ul)) (152) 

This shows that the weakened version of the upper bound obtained in (I120p is indeed tight 
and a binary auxiliary random variable T with uniform distribution over {0, 1} is sufficient 
to attain this symmetric-rate upper bound. 



10 Evaluation of the Cover-Leung Achievable Rate Re- 
gion 

For completeness we will also obtain a simple characterization of the Cover-Leung inner 
bound for our binary additive noisy MAC-FB. For this purpose, we follow a two-step ap- 
proach. In the first step, we first obtain an outer bound on the achievable rate region in 
terms of two variables (ui,u 2 ). In the second step, we specify an input distribution, as a 
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function of (1x1,112), which achieves the outer bound. We therefore arrive at an alternate 
characterization of the Cover-Leung achievable rate region in terms of the variables (ui,u 2 ). 
The Cover-Leung achievable rate region [3] is given as, 

CC= [(Ri,R 2 ) : Ri < I{X X ;Y\X 2 ,T) (153) 
R 2 <I(X 2 ;Y\X 1 ,T) (154) 
Ri + R 2 <I(X 1 ,X 2 ;Y)} (155) 

where the random variables (T, X±,X 2 ,Y) have the joint distribution, 

pit, xx, x 2 , y) = p(t)p(xx\t)p(x 2 \t)p(y\xi, x 2 ) (156) 

and the random variable T is subject to a cardinality constraint of \T\ < min(|Ai||A2| + 1, 
\y\ + 2). For the binary, additive noisy MAC in consideration, the constraints in ( 11 53f) - ( TX55T) 
become, 

Ri < \HiXx\T) (157) 

R2 < \H{X 2 \T) (158) 
Ri + R 2 <I(X 1 ,X 2 ;Y) (159) 

We will first obtain an outer bound on the region specified by ( 1157p -( fT59l) in terms of two 
variables (ui,u 2 )- For every pair (ui,u 2 ), we will then specify an input distribution which 
will attain this outer bound. Note that the three constraints f ll57l) -( fT59l are of similar form 
as in the case of T>B^ C and T>B^ C , and we proceed in a similar manner to obtain upper 
bounds on the three terms above in terms of U\ and u 2 as, 

Ri < \k<P(2 Ui )) (160) 
R 2 < \h(<p(2u 2 )) (161) 
R 1 + R 2 <h( LlI^A) (i 62) 



where the variables (ui,u 2 ) belong to the set S defined in (I133p . Hence, an outer bound on 
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the rate region specified by fll57l) - (ll59p is given as O, where 
O 



|J |(i2i,i22):i2i<^(0(2ui)) 
(ui,u 2 )SiS L 



R 2 < l -h(<p(2u 2 )) 

R 1 + R*<h( 1 - f{2 ^ u > ) ) } (163) 



Let (ui,u 2 ) be any arbitrary pair which belongs to S. Consider an input distribution for 
which \T\ = 2, and T is uniform over {0, 1} and, 

Po=Pi = \ (164) 

<?io = 1 - q n = 0(2«i) (165) 

920 = 1 - <?2i = 0(2m 2 ) (166) 

For this input distribution, we obtain the following exact equalities 

H(X 1 \T) = h(<j ) (2u 1 )) (167) 

H(X 2 \T) = h((j){2u 2 )) (168) 

I(X 1 ,X 2 -,Y) = h( 1 - f ^ 2 ^ ) (169) 



We have thus shown that the outer bound we obtained on the achievable rate region in terms 
of (ui,u 2 ) can be attained by a set of input distributions for which the involved auxiliary 
random variable T is binary and uniform. This in turn implies that a binary and uniform 
random variable T is sufficient to characterize the entire Cover-Leung achievable rate region 
for the binary additive noisy MAC-FB. By varying over all such input distributions, or 
equivalently, by varying (ui,u 2 ) in the set S, we obtain the entire Cover-Leung achievable 
rate region. We should remark here that when evaluating the T>Bpc bound in the previous 
section for Z — X\ and Z = X 2 , it was not necessary to specify the distribution which 
achieves the bound, since it was an outer bound. On the other hand, when evaluating the 
Cover-Leung bound, since it is an achievability, it is necessary to give a distribution which 
achieves the bound. 

The dependence balance bounds corresponding to the parallel channel choices Z — X\ 
and Z = X 2 , along with the cut-set upper bound and the Cover-Leung achievable rate region 
are shown in Figure 4. It is interesting to note that our bound improves upon the cut-set 
bound at all points where the Cover-Leung achievable rate region is strictly larger than the 
capacity region without feedback. In other words, our bound improves upon the cut-set 
bound at all points where feedback increases capacity. 
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We should remark that our choices of parallel channels; namely, Z = X\ and Z = X 2 are 
the simplest ones which ensure that I(Xi; X 2 \Y, Z,T) = but they yield fixed information 
leaks. We believe that by a more elaborate choice of a parallel channel, i.e., by carefully 
selecting a parameterized parallel channel p + (z\xi, x 2 , y, t) such that I{X\\ X 2 \Y, Z, T) = 0, 
one would still be able to restrict the input distributions to a conditionally independent form 
and then optimize the parameters of the parallel channel to minimize the information leak 
terms. This approach can potentially improve upon our outer bound. 



11 The Capacity Region of the Binary Erasure MAC- 



The capacity region of a class of discrete memoryless MAC-FB was characterized in [9] by 
establishing a converse and it was shown to be equal to the Cover-Leung achievable rate 
region. This class of channels satisfy the property that at least one of the channel inputs say 
Xx, can be written as a deterministic function of the other channel input X 2 and the channel 
output Y. The binary erasure MAC, where Y = X\ + X 2 , falls into this class of channels. 
In addition, the binary erasure MAC-FB is the noiseless version of the binary additive noisy 
MAC-FB studied in this paper. 

Willems showed in [12] that a binary selection of auxiliary random variable is sufficient 
to obtain the sum-rate point of the capacity region of the binary erasure MAC-FB. In this 
section, we will show that by using our results for composite functions which were presented 
in previous sections, it is possible to obtain all points on the boundary of this capacity region 
using a binary auxiliary random variable. The feedback capacity region of this channel is 
given by the Cover-Leung achievable rate region given in (I153p - (I155I) which can be simplified 
for the binary erasure MAC-FB as, 



We obtain three upper bounds on the expressions appearing in the bounds (11 701) - ( 1X721) . From 
( |62l) . we have, 



FB 



R 1 <H(X 1 \T) 
R 2 < H{X 2 \T) 
Ri + R2< H(Y) 



(170) 
(171) 
(172) 



H{Xi\T) < /i(0(2ui)) 



(173) 



Similarly, we also have 



H(X 2 \T) <h(<f>(2u 2 )) 



(174) 
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We now obtain an upper bound on H(Y), by first noting that, 

H(Y) = h^(P Y (0),P Y (l),Py(2)) (175) 

where 

Py(0) = J2PtHtq2t (176) 

t 

P Y {l) = ^Pt{qit + q2t-2q lt q 2t ) (177) 
t 

P Y {2) = Y,Pt{l-qn){l-q2t) (178) 

t 

Now, we use the following inequality established in [12], 

fc( 3 ) (a, b, c) = i/i (3) (a, b, c) + i/i (3) (c, b, a) (179) 
</> {3) fe&,^l (180) 



2 ' ' 2 

= /i(6) + l-6 (181) 

where (1180p follows by the concavity of the entropy function and by the application of 
Jensen's inequality [13]. Using ( 118ip and continuing from (I175j) . we obtain 

H(Y) = h®(P Y (0),P Y (l),P Y (2)) (182) 
<h(P Y (l)) + l-P Y (l) (183) 
= + 1 - u (184) 

where u is defined in fl55l) . Using (I173p . (I174p and (I184p . we can write an outer bound 0\ 
on the capacity region as follows, 

0i= |J 1 {u 1 ,u 2 ,u) (185) 

where 

Oi(tti,« 2 ,«) = {(Ri,R 2 ) ■ i?i < h{<P{2 Ul )) 

R 2 < h{4>{2u 2 )) 

R 1 + R 2 < h(u) + 1 - «} (186) 
and the set "P is defined in (IllOp . We will now obtain a simpler characterization of 0\ in 
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terms of two variables (ui,u 2 ) by showing that 0\ = 2 , where, 



2 = |J 2 ( Ul ,u 2 ) (187) 
(«i,u2)e<s 



where 



2 (ui,ti2) = {(^1,^2) : i?i < /i(^(2ui)) 

R 2 < h((j)(2u 2 )) 

Ri + R2< /i(/(2«i,2«2)) + l-/(2wi,2« 2 )} (188) 

The inclusion 2 C Ox is straightforward by forcing u = f(2ui,2u 2 ) in C^. We will now 
show that 0\ C 2 . For this purpose, we will need the following lemma. 

Lemma 4 TTie junction 

H(s) = h(s) + l-s (189) 

is concave in s for s G [0, 1] and takes its maximum value at s = |. Moreover, the function 
(j,(s) is increasing in s for s G [0, |] and decreasing in s for s G [|, 1]. 

The proof of this lemma follows from the fact that both h(s) and —s are concave in s. 

Now consider any arbitrary triple {ui,u 2 ,u) G V. We can classify any such triple into 
one of the following cases: 

1. If f(2ui,2u 2 ) < u < |: for any such (ui,u 2 ,u), there exists a pair (vli,u 2 ), such that 

ui < iii < i (190) 
u 2 < u 2 < i (191) 
u = /(2ui,2ii2) (192) 

One such pair (fZi,^) can be obtained as follows. Using the fact that for a fixed ui, 
f(2ui,2u 2 ) is increasing in u 2 , we select u\ = u\ and solve for u 2 < u 2 < \ for which 
f(2ui, 2u 2 ) = u. The required u 2 is obtained as, 

« 2 = 7 U-TT— rA (193) 



4 V (l-4ui) 
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For such a pair (ui,u 2 ), the following inequalities hold, 

h(<p(2 Ul )) = /i(0(2mx)) (194) 

h{4>{2u 2 )) < h((f)(2u 2 )) (195) 

h(u) + 1 - u = h(f(2u h 2u 2 )) + 1 - f(2u 1 , 2u 2 ) (196) 

2. If f(2u\, 2u 2 ) < ~ < u < 1 — («i + M2), then we have by Lemma 4, 

Ku) + l- u <h(^\+l-± (197) 



3 
2 



:i98) 



Now consider the pair (■u 1 ,-u 2 ) = (2, |), for which we have f(2ui,2u 2 ) = ~. Hence we 
have that, 

h{(j){2ux)) < h^ux)) = 1 (199) 

h((f)(2u 2 )) < h{(j){2u 2 )) = 1 (200) 

h(u) +l-u< h(f(2u 1 , 2u 2 )) + 1 - f(2u 1 , 2u 2 ) = - (201) 

We have thus shown that for any triple (ui,u 2 ,u), there exists a pair (ui,u 2 ), such that 
Oi(ui,u 2 ,u) C ^(-ux, M2), which in turn implies that 0\ C C 2 , and consequently Ci = 2 . 
Hence, we have an outer bound on the capacity region as given by 2 . 

The outer bound 2 is evaluated over the set of pairs (^1,^2) such that ui,u 2 G [0, 7]. 
For any such arbitrary pair (u\, u 2 ), an input distribution which achieves the set of rate pairs 
specified by 02(^1,^2) is obtained by selecting |T| = 2, and 

Po = Pi = \ (202) 
9io = 1 - 9n = 0(2«i) (203) 
920 = 1 - 92i = 0(2w 2 ) (204) 

The set of rates achievable by the distribution specified in ( 1202p -( j204l) are obtained as, 

R x < E{X X \T) = /i(0(2ux)) (205) 
R 2 < H(X 2 \T) = h((f>(2u 2 )) (206) 
Ri + R 2 < H(Y) = h(f(2u u 2« 2 )) + 1 - /(2«i, 2u 2 ) (207) 

This shows that the capacity region of binary erasure MAC-FB can be obtained by a 
binary and uniform selection of the auxiliary random variable T. The capacity region of 
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the binary erasure MAC with and without feedback and the cut-set bound are illustrated 
in Figure 5. It was shown in [12] that the sum-rate point on the boundary of the capacity 
region lies strictly below the "total cooperation" line. This is equivalent to saying that the 
cut-set bound is not tight for the sum-rate point. From our result, it is now clear that the 
cut-set bound is not tight for asymmetric rate pairs either. In fact, it is not tight at all 
boundary points where feedback increases capacity. 

Moreover, our result also shows that a simple selection of binary and uniform T is suf- 
ficient to evaluate the boundary of the capacity region of binary erasure MAC-FB. Simple 
feedback strategies for a class of two user MAC-FB were developed in [4]. It was shown 
that for the binary erasure MAC, these feedback strategies yield all rate points for a binary 
selection of the auxiliary random variable T. Thus, our result shows that these feedback 
strategies are indeed optimal for the binary erasure MAC-FB and yield all rates on the 
boundary of its feedback capacity region. 

12 Conclusions 

In this paper, we obtained a new outer bound on the capacity region of a MAC-FB by using 
the idea of dependence balance. We considered a binary additive noisy MAC-FB for which 
it is known that feedback increases capacity but the feedback capacity region is not known. 
The best known outer bound on the feedback capacity region of this channel was the cut-set 
bound. We used the dependence balance bound to improve upon the cut-set bound at all 
points in the capacity region of this channel where feedback increases capacity. Our result 
is somewhat surprising once it is realized that the channel we considered in this paper is the 
discrete version of the two-user Gaussian MAC-FB considered by Ozarow in [2] where the 
cut-set bound was shown to be tight. 

Our outer bound is difficult to evaluate due to an involved auxiliary random variable 
T. For binary inputs, the cardinality bound on T is \T\ < 7 which makes it intractable to 
evaluate the outer bound. We overcome this difficulty by making use of composite functions 
and their properties to obtain a simple characterization of our bound. As an application 
of the properties of the composite functions developed in this paper, we are also able to 
completely characterize the Cover-Leung achievable rate region for this channel. 

The capacity region of the binary erasure MAC-FB is known and it coincides with the 
Cover-Leung achievable rate region. Although the capacity region is known in principle, it is 
not known how to compute the entire region, the difficulty arising again due to the involved 
auxiliary random variable. We again make use of the composite functions to give an alternate 
characterization of the capacity region of the binary erasure MAC-FB. In addition, we go 
on to show that a binary and uniform auxiliary random variable selection is sufficient to 
evaluate its feedback capacity region. 
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Figure 4.1: Illustration of our bounds for the capacity of binary additive noisy MAC-FB. 
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Figure 4.2: An enlarged illustration of the portion of Figure 4.1 where feedback increases 

capacity. 
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Figure 5.1: Illustration of the capacity region of binary erasure MAC-FB. 
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